Accelerating Point-Based POMDP Algorithms through Successive Approximations of the Optimal Reachable Space

Point-based approximation algorithms have drastically improved the speed of POMDP planning. This paper presents a new point-based POMDP algorithm called SARSOP. Like earlier point-based algorithms, SARSOP performs value iteration at a set of sampled belief points; however, it focuses on sampling near the space reachable from an initial belief point under the optimal policy. Since neither the optimal policy nor the optimal reachable space is known in advance, SARSOP builds successive approximations to it through sampling and pruning. In our experiments, the new algorithm solved difficult POMDP problems with more than 10,000 states. Its running time is competitive with the fastest existing pointbased algorithm on most problems and faster by many times on some. Our approach is complementary to existing pointbased algorithms and can be integrated with them to improve their performance.